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ABSTRACT 


AUTHOR:  Mr. Richard  Swain 

TITLE:  Changes  In  Instructional  System  Design  (ISD):  Improving  Training  Product 

Delivery  To  United  States  Army  Soldiers 

FORMAT:  Strategy  Research  Project 

DATE:  18  March  2005  PAGES:  48  CLASSIFICATION:  Unclassified 

The  purpose  of  this  study  is  to  identify  methods  that  could  speed  up  the  instructional 
system  design  process  currently  used  by  the  U.  S.  Army.  The  current  Army  Instructional 
System  Design  process  is  the  Systems  Approach  to  Training  (SAT),  a  thirty-year-old  process. 
SAT  is  an  industrial  age  process  being  applied  to  an  information  age  Army.  This  study 
surveyed  Army  training  development  experts  about  what  they  see  as  the  significant  challenges 
in  the  SAT  process.  This  Strategic  Research  Project  describes  the  strengths  and  weaknesses 
of  the  current  process.  The  ultimate  goal  is  to  give  training  leaders  recommendations  that,  if 
implemented,  will  make  a  significant  impact  on  the  effectiveness  of  the  SAT  process. 
Recommend  in  the  short-term  that  TRADOC  immediately  hire  and  train  more  training 
developers.  This  is  regardless  of  the  ISD  system  that  is  eventually  chosen.  TRADOC  lacks  a 
sufficient  number  to  accomplish  the  mission.  TRADOC  Schools  should  form  the  Training 
Developers  into  multi-disciplinary  teams  and  use  automated  tools  that  will  speed  the  SAT 
process  (e.g.  Designers  Edge).  Additionally,  the  report  recommends  an  emphasis  on  the 
evaluation  of  the  SAT  process  as  a  part  of  the  Quality  Assurance  Office  mission.  To  develop  a 
long-term  solution  TRADOC  should  immediately  begin  experimenting  with  Rapid  Prototyping  in 
a  head-to-head  contest  with  the  SAT  process. 
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PREFACE 


Understanding  Instructional  System  Design  as  it  is  practiced  today  throughout  the  Army 
will  give  leaders  the  information  needed  to  improve  the  process.  The  greatness  of  our  Army 
Training  Programs  is  exemplified  in  the  highly  effective  and  professional  force  that  the  nation 
currently  enjoys.  The  improvement  that  might  be  realized  by  a  deep  understanding  of  the 
process  is  worth  the  effort  if  it  saves  but  one  Soldier's  life  due  to  improved  training  and 
education  products.  The  purpose  of  the  research  is  to  describe  the  current  situation  “with  the 
bark  on,”  as  our  current  Secretary  of  Defense  Donald  Rumsfeld  likes  to  say.  Additionally,  it  will 
provide  training  leaders  actionable  policy  recommendations  to  increase  SAT  effectiveness. 
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CHANGES  IN  INSTRUCTIONAL  SYSTEM  DESIGN  (ISD):  IMPROVING  TRAINING  PRODUCT 
DELIVERY  TO  UNITED  STATES  ARMY  SOLDIERS 


T raining  development  is  a  vital  component  of  the  mission  of  T raining  and  Doctrine 
Command  (TRADOC)  to  prepare  the  Army  for  war.  Preparation  of  the  Army  for  war  is  the 
responsibility  of  every  civilian  and  soldier  in  management  and  training-related  roles  in  the 
TRADOC  HQ,  schools,  and  supporting  contractor  offices.  One  means  of  creating  training 
products  for  use  in  the  field  is  Instructional  Systems  Design  or  ISD.  ISD  is  a  widely  used 
methodology  for  developing  training  programs.  The  ISD  field  of  study  in  education  uniquely 
combines  disciplines  from  business  to  psychology.  The  process  and  procedure  have  various 
monikers:  Instructional  System  Design  &  Development  (ISDD),  the  Systems  Approach  to 
Training  (SAT)  (the  Army’s  chosen  nomenclature),  or  just  Instructional  Design  (ID).  This 
approach  is  a  repeatable  step-by-step  process  to  develop  training  materials  and  evaluate  the 
effectiveness  of  an  instructional  event.  Specifically,  "ISD,  evolved  from  post-World  War  II 
research  in  the  United  States  military  to  find  a  more  effective  way  to  create  training  programs."1 
The  Army  adopted  its  current  instructional  system  design,  the  SAT  process  in  the  1 970s.2 

While  useful  when  adopted,  much  has  happened  in  the  last  thirty-five  years.  The  velocity 
of  recent  technological  change  that  has  occurred  throughout  society,  for  example,  has  resulted 
in  the  "Rapid  acquisition  and  fielding  of  new  technology,  spiral  integration  of  system  of  systems 
innovations  and  exploitation  of  best  business  practices  that  demand  new  ways  of  thinking  and 
new  ways  of  managing  change  at  every  level  of  Army  leadership."3 

The  technological  advantage  the  U.S.  Armed  Forces  possess  is  very  evident  in  the  War 
on  Terrorism,  especially  the  war  with  Iraq.  Despite  these  successes,  commanders  have 
indicated  in  after  action  reports  that  training  needs  improving.  For  example,  the  Third  Infantry 
Division  After  Action  Report  stated: 

FBCB2  as  a  command  and  control  medium  was  extremely  useful  and  effective.  It 
provided  unprecedented  situational  understanding  for  all  commanders  and 
command  posts.  It  also  allowed  the  BCT  to  forward  graphics,  messages,  and 
fragmentary  orders  (FRAGOs)  across  distances  that  FM  could  not  cover. 
Limitations  included  the  lack  of  training  received  on  all  facets  of  the  system,  poor 
performance  of  electronic  messaging,  and  a  limited  number  of  systems  that 
allowed  fielding  to  company  commander  and  executive  officer  level  in  maneuver 
units  only.  To  correct  these  shortfalls  Blue  Force  Tracking  (BFT)  should  increase 
its  messaging  capabilities  and  be  fielded  to  every  vehicle  in  the  division.4 


The  need  for  faster  response  times  in  creating  training  for  soldiers  became  obvious  as  the 
enemy  attacked  using  asymmetric  means  and  methods.  For  instance:  during  the  war,  a  mine 
identification-training  product  was  needed  immediately.  The  normal  SAT  process  would  have 


taken  1 8  months  to  complete  the  project.  The  Engineer  School  produced  the  product  in  30 
days.  This  abbreviated  process  worked,  but  the  Engineer  School  methodology  is  neither 
sanctioned  nor  approved  in  the  current  TRADOC  Regulation  350-70,  the  guidance  for  the  SAT 
process. 

Nowhere  is  this  new  thinking  more  important  than  in  the  Army's  development  of  training, 
and  especially,  ISD.  As  a  result,  senior  Army  Leaders  responsible  for  training  are  asking 
whether  a  thirty-year-old  instructional  system  design  process  can  be  effective  for  today's  Army? 
As  Major  General  Barrett  observed  during  the  March  2003  Distributed  Learning  Conference, 
"We  can  no  longer  take  1 8  months  to  produce  the  training  products  the  Army  needs  to  be 
effective  in  combat.  When  changes  in  weapons  systems  occur  at  an  increasing  rate  of  speed."5 
Quite  specifically,  Major  General  Barrett  questioned:  "Does  the  SAT  process  meet  the  needs  of 
the  force  that  must  be  able  to  change  dramatically  over  the  next  five  years  to  meet  the  training 
challenges  of  new  weapon  systems,  new  tactics  and  doctrine?'6 

Indeed,  changes  in  ISD  over  the  last  thirty  years  have  been  considerable.  ISD  models 
like  Rapid  Prototyping,  Tutorial,  Experiential  and  Elaboration  have  been  effective  in  producing 
training  products  for  the  civilian  market.  We  should  expect  to  identify  differences  in  the 
instructional  system  design  process  that  worked  in  the  1 970s  compared  to  the  instructional 
system  design  process  used  in  2005.  Changes  in  techniques  of  teaching  and  learning  also 
affect  the  instructional  system  design  process.  Better,  faster,  cheaper  ways  to  create 
instructional  products  for  the  Army  might  be  available.  The  SAT  process  might  or  might  not  be 
the  correct  process  to  meet  the  goals  of  a  21  st  Century  Army.  Given  these  circumstances,  we 
should  carefully  study  the  Army’s  current  instructional  design  and  delivery.  A  systematic 
analysis  of  the  characteristics  of  the  instructional  system  design  process  in  the  U.  S.  Army 
logically  follows.  This  description  and  analysis  could  provide  a  significant  part  of  the  answer  to 
the  question:  What  factors  can  influence  the  reduction  of  process  time  in  the  U.S.  Army? 

BACKGROUND  OF  THE  STUDY 

The  Army’s  training  development  process  is  based  on  a  systematic,  spiral  approach  to 
making  collective,  individual,  and  self-development  training  decisions  for  the  total  Army.  The 
process  determines  whether  the  training  is  needed;  what  is  to  be  trained;  who  gets  the  training; 
how,  how  well,  and  where  the  training  is  presented;  and  the  training  support/resources  required 
to  produce,  distribute,  implement,  and  evaluate  those  products.  The  process  involves  five 
training  related  phases:  analysis,  design,  development,  implementation,  and  evaluation.7  Table 
1  shows  the  accepted  acronym  for  this  process,  which  is  ADDIE. 
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A 

ANALYSIS 

D 

DESIGN 

D 

DEVELOPMENT 

1 

IMPLEMENTATION 

E 

EVALUATION 

TABLE  1:  (ADDIE)  INSTRUCTIONAL  DESIGN  MODEL8(CLARK,  1999) 


This  basic  ADDIE  process  was  adapted  and  enlarged.  The  model,  graphically  illustrated 
below,  called  the  Interservice  Procedures  for  Instructional  Systems  Development  (IPISD),  was 
intended  for  use  with  large-scale  instructional  development.  The  IPISD  is  the  precursor  of  the 
SAT. 


FIGURE  1 :  INTERSERVICE  PROCEDURES  FOR  INSTRUCTIONAL  SYSTEMS 
DEVELOPMENT  (IPISD) 


The  background  to  the  problem  of  whether  the  current  ISD  is  adequate  is  best  illustrated 
by  actions  of  the  1 990s  which  included  the  business  process  of  right  sizing,  downsizing, 
concentration  on  the  core  competency  of  the  organization,  and  contracting  out  non-core 
competencies  of  the  business  enterprise.  The  Training  Development  community  within 
TRADOC  was  drastically  reduced  in  the  1 990s  (41  %  reduction  in  civilian  personnel  from  FY91 
to  FY01)9.  Moreover,  during  this  period,  TRADOC's  workload  had  grown  because  of  new 
missions  since  1 995.  Additionally,  a  Government  Accounting  Office  (GAO)  investigation  of 
TRADOC's  readiness  to  perform  its  mission  noted  that  between  1 995  and  2001  the  number  of 
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essential  TRADOC  personnel  assigned  to  develop  Distance  Learning  Products  conduct  this 
mission  declined  from  80%  of  authorized  positions  in  1995  to  71%  in  2001 .  Worse  still  the  GAO 
Report  specifically  indicated  that  30  reporting  units  of  TRADOC  reported  the  lowest  readiness 
status  due  to  personnel  shortages.  Without  adequate  number  of  training  development 
personnel  the  SAT  process  is  difficult  to  implement.10 

CURRENT  SYSTEM 

Management  at  all  levels  needs  both  to  understand  the  SAT  process  and  to  ensure  its 
efficient  implementation.  Success  in  doing  so  will  save  scarce  resources:  personnel,  time,  and 
unnecessary  product  development  dollars.  The  SAT  is  best  described  as  a  diagram.  Figure  2 
graphically  portrays  the  cyclical  nature  of  the  SAT  process.  Figure  3  depicts  how  each  phase  of 
the  SAT  model  builds  upon  preceding  phases.  Although  the  phases  build  upon  each  other,  it  is 
important  to  remember  that  the  normal  training  development  process  for  a  new  training 
development  requirement  begins  with  evaluation  (a  perceived  training  requirement)  and 
proceeds  with  other  analyses,  followed  by  design,  development,  and  implementation  of  the 
training/training  product.11  At  the  same  time,  each  phase  and  product  must  meet  minimum 
essential  requirements.12  Not  all  phases  have  to  be  followed  in  order;  each  phase  can  be 
entered  individually  as  needed  for  revisions.  The  process  is  a  continuous  series  of 
development,  revision,  and  implementation  events. 

Evaluation  permeates  all  phases  and  is  the  cement  that  ensures  all  training  and  training 
products  are  effective  in  producing  trained  units  and  soldiers.  Products  are  evaluated  either 
formally  (i.e.,  product  validation)  or  informally  to  determine  currency,  efficiency,  and 
effectiveness,  followed  by  revisions  as  required. 
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External  Evaluation 


FIGURE  2:  SYSTEMS  APPROACH  TO  TRAINING  (SAT)  PROCESS 


FIGURE  3:  SYSTEMS  APPROACH  TO  TRAINING  (SAT)  PROCESS 
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TRADOC  Regulation  350-70  provides  guidance  on  implementing  SAT.  TRADOC 
Regulation  350-70  states  “Bottom  Line:  Proper  use  of  the  SAT  process  will  result  in 
development  of  essential  training/products  and  conservation  of  resources”.  The  regulation 
recognizes  the  methodical  approach  inherent  in  SAT  and  identifies  many  techniques  to  manage 
Training  Development  efficiencies: 


1 .  Revise/Update  analysis  and  design  data  when  changes  occur  instead  of 
going  through  the  entire  analysis,  design,  and  development  processes 

2.  Redesign/revise  existing  products  versus  begin  new  product  development 
when  possible. 

3.  Set  up  a  dedicated  team  composed  of  a  team  leader  (Instructional  Systems 
Specialist);  appropriate  AC  and  RC  SME(s);  and  editor,  visual  information 
specialist,  evaluation  personnel,  and  training  specialist(s)  as  necessary. 

4.  Train  team  members  in  the  SAT  process. 

5.  Keep  the  same  team  throughout  the  development/revision. 

6.  A  dedicated  team  can  perform  analysis,  design,  and  development  functions 
more  efficiently  and  save  more  staffing  time  than  developers  working  in 
various  divisions  doing  various  jobs. 

7.  Use  TD  automation  tools  as  they  become  available  to  meet  SAT  process  and 
management  requirements,  e.g.,  ASAT  suite 

8.  The  TD  automated  workload  management  database  (currently  ASAT 
[Resource  Management  menu  option])  to  plan  and  prioritize  workload  and 
assist  in  personnel  management 

9 .  Contracting  for  additional  or  specialized  TD  support. 

10.  Access  Reserve  Component  assets  when  possible:  Title  10  (coordinate  with 
Office  of  the  Chief,  Army  Reserve  [OCAR]  and  the  National  Guard  Bureau 
[NGB])  and  Title  32  (coordinate  with  state  Adjutant  Generals)  for  TD  support 
of  TATS  Courses 

1 1 .  Develop  products  in  a  structured  writing  format  when  feasible13 


Despite  these  efforts  and  shortcuts,  the  current  SAT  process  is  cumbersome,  highly 
detailed,  and  rule  intensive.  For  instance,  according  to  planning  figures,  8655  man  years  were 
needed  in  2002  to  implement  the  SAT  process  for  all  schools.  However,  due  to  personnel 
shortages,  only  3244  man  years  were  available,  resulting  in  a  backlog  of  541 1  man  years  of 
Training  Development  work  for  2002  alone. 
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Given  the  lack  sufficient  personnel  necessary  to  use  the  SAT  system  and  the  lengthy, 
cumbersome  methodology,  should  TRADOC  retain  the  process?  In  an  initial  effort  to  answer 
that  question,  the  author  asked  Dr.  Robert  K.  Branson,  creator  of  the  process,  whether  SAT  is 
still  an  appropriate  method  of  instructional  design.  Briefly,  Dr.  Branson  answered  that  he  was 
amazed  with  the  universal  acceptance  of  the  process,  which  he  believed  to  be  a  major  strength. 
But,  Dr.  Branson  also  noted  that  many  paths  can  get  you  to  your  goal,  as  long  as  you  know 
what  you  want  at  the  end.14 

As  part  of  the  interchange,  the  author  asked  Dr.  Branson  about  the  importance  of  the 
quality  assurance  portion  of  the  SAT  process.  Dr.  Branson's  comments  both  reinforced  the  SAT 
model  and  gave  the  Army  leeway  to  find  other  methods  to  develop  training  materials  that 
soldiers  need  to  increase  their  combat  readiness  and  enhance  soldier  survival.  Doctor  Branson 
responded: 

There  is  no  known  way  to  have  an  effective  quality  control/quality  assurance 
program  without  collecting  the  necessary  product  and  process  evaluation  data. 
Without  quality  control  and  quality  assurance  functions,  there  can  be  no  effective 
control  function  in  a  system.  Without  a  control  function,  there  can  be  no  system. 

As  you  all  know,  evaluation  data  can  be  used  for  assessing,  improving,  and 
establishing  the  value  of  problems  in  any  instructional  program.  Evaluation  data, 
in  the  hands  of  the  inspector  general,  can  also  be  used  as  a  vehicle  to  indict 
practitioners.  When  evaluation  or  quality  control  data  are  used  as  a  club,  the 
normal  human  response  is  to  eliminate  the  evaluation  function,  or  the  evaluator  -- 
or  both.15 

In  a  final  observation,  Branson  described  precisely  what  happened  to  the  Quality 
Control/Quality  Assurance  function  in  the  United  States  Army  in  1 992.  The  Directorate  of 
Evaluation  and  Standardization  (DOES)  was  the  organizational  predecessor  of  the  Quality 
Assurance  Office.  The  Directorate  of  Evaluation  and  Standardization  converted  from  the  role  of 
evaluation  designed  to  assist  the  system  to  a  form  of  inspection.  Many  organizations  in 
TRADOC  viewed  DOES  as  a  threat,  so  many  of  them  simply  disbanded  the  organizations. 

Some  Directorates  of  Evaluation  and  Standardization  survived  due  to  the  method  of  evaluation 
practiced  and  their  offices  contribution  to  the  school.  Some  schools  renamed  the  DOES  to 
camouflage  the  function  in  other  divisions  or  directorates  within  the  organization.  This  was  done 
to  save  the  capability  within  some  organization.  Some  schools,  like  the  Air  Defense  Artillery 
School,  eliminated  the  function  completely.  However,  what  is  significant  to  understand  is  the 
Evaluation/Quality  Control/Assurance  function  is  critical  to  the  SAT  process  to  be  effective. 
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AN  ANALYSIS  OF  SAT 


To  determine  whether  SAT  continues  to  meet  the  needs  of  the  force  or  will  need  to  be 
adapted  or  replaced,  first  requires  an  assessment  of  the  current  SAT  process.  To  assist  in  the 
assessment,  the  author  received  permission  to  ask  the  Civilian  Training  Leaders  in  the  U.S. 
Army  what  they  thought  about  SAT  and  howto  improve  it.16  The  questionnaire  sought  to  draw 
insights  into: 

•  The  relationship  of  abbreviating  steps  in  SAT  process; 

•  The  Army  Instructional  System  Design  process  as  it  is  implemented  currently; 

•  The  challenges  facing  Army  trainers  in  implementing  the  SAT  process; 

•  Any  correlation  between  the  abbreviations  of  the  steps  in  the  SAT  process  and 
increased  speed  of  training  product  delivery  to  Soldiers. 

DISCUSSION  (CURRENT  SYSTEM  INADEQUATE) 

Only  23.1%  of  the  Civilian  Training  Leaders  rated  SAT  as  “Very  Effective”.  Another  34% 
indicated  that  SAT  is  “Somewhat  Effective”.  A  further  16.9%  responded  that  SAT  is  “Neither 
Effective  or  Ineffective”.  An  additional  8.7%  of  Civilian  Training  Leaders  indicated  that  SAT  is 
“Somewhat  Ineffective”.  Finally  5.6%  of  respondents  rated  SAT  as  “Very  Ineffective”.  That  only 
57%  of  the  Civilian  Training  Leaders  believe  SAT  is  effective  in  meeting  the  Instructional 
Development  needs  of  the  U.S.  Army,  is  not  a  ringing  endorsement  of  the  process.  The  table 
below  compares  the  percentages  for  each  of  the  steps: 


SAT  Steps 

Analysis 

Design 

Development 

Implementation 

Evaluation 

Total 

Very 

Effective 

27.8% 

20% 

21.1% 

23.3% 

23.3% 

m 

Somewhat 

Effective 

31.1% 

27.8% 

40% 

40% 

31.1% 

34% 

Neither 
Effective  or 
Ineffective 

15.6% 

22.2% 

16.7% 

12.2% 

17.8% 

■ 

Somewhat 

Ineffective 

8.9% 

13.3% 

5.6% 

8.9% 

6.7% 

8.7% 

Very 

Ineffective 

5.6% 

4.4% 

5.6% 

3.3% 

8.9% 

5.6% 

No 

Response 

11% 

12.3% 

11% 

12.3% 

12.2% 

1 1 .7% 

Total 

100% 

■ebshb 

100.% 

100% 

100% 

100% 

TABLE  2:  SYSTEMS  APPROACH  TO  TRAINING  COMPARISON 
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Identifying  overall  satisfaction  or  dissatisfaction  with  SAT  is  only  a  first  step  in 
understanding  the  strengths  and  weaknesses  of  the  process.  Of  greater  importance,  perhaps, 
is  understanding  why  Civilian  Training  Leaders  think  the  process  is  challenged?  To  identify 
possible  areas  of  weaknesses,  respondents  were  asked:  “In  your  opinion  what  are  the  top  three 
challenges  facing  the  SAT  (Systems  Approach  to  Training)  process  in  the  United  States  Army?” 
The  top  three  challenges  indicated  by  these  leaders  were,  first,  the  current  SAT  process  is  fine, 
but  military  leaders  do  not  know  how  to  manage  the  process.  Second,  there  are  not  enough 
Training  Developers  to  implement  the  process.  And,  third,  there  aren’t  enough  Training  Leaders 
who  understand  the  SAT  Process.17 

The  below  responses  indicate  an  overwhelming  perception  of  the  need  for  more 
personnel  and  funding  for  SAT.  Tables  3  and  4,  respectively,  graphically  portray  the  data. 


Frequency 
Percent 
Valid  Percent 
Cumulative  Percent 

Valid 

"Very  Insufficient" 

43 

50.0 

51.8 

51.8 


"Somewhat  Insufficient" 
28 

32.6 

33.7 
85.5 


"Neither  Sufficient  or  Insufficient" 
3 

3.5 

3.6 
89.2 


"Somewhat  Sufficient" 
8 

9.3 

9.6 


9 


98.8 


"Very  Sufficient" 
1 

1.2 

1.2 

100.0 


Total 

83 

96.5 

100.0 


Missing 

System 

3 

3.5 


Total 

86 

100.0 


TABLE  3:  PERSONNEL  SUFFICIENT  TO  MEET  SAT  REQUIREMENTS 


Frequency 
Percent 
Valid  Percent 
Cumulative  Percent 

Valid 


10 


"Very  Insufficient" 

40 

46.5 

48.2 

48.2 


"Somewhat  Insufficient" 
28 

32.6 

33.7 
81.9 


"Neither  Sufficient  or  Insufficient" 
4 

4.7 

4.8 

86.7 


"Somewhat  Sufficient" 
10 
11.6 
12.0 

98.8 


"Very  Sufficient" 
1 

1.2 

1.2 

100.0 


Total 

83 

96.5 

100.0 


Missing 

System 

3 

3.5 


Total 

86 

100.0 
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TABLE  4:  SUFFICIENT  FUNDING  FOR  THE  SAT  PROCESS 


Forty  three  percent  of  the  Civilian  Training  Leaders  agreed  that  “Current  SAT  process  is 
fine,  but  military  leaders  do  not  know  how  to  manage  the  process”.  The  revolving  and  rapid 
turnover  of  Directors  of  T raining  at  TRADOC  Schools  is  a  possible  reason  why  military  leaders 
are  not  able  to  manage  the  SAT  process.  Interviews  with  Senior  Civilian  Training  Leaders 
indicate  the  lack  of  personnel  and  funding  sets  the  system  up  to  fail  under  any  leadership.18 
The  Military  Leader  is  placed  into  a  difficult  situation  without  adequate  training  and  experience. 

In  general,  12.5%  of  Training  Leaders  stated  that  the  SAT  process  is  not  adequate  for 
today's  training  needs:  the  speed  of  change  in  weapons  systems  and  training  outpace  the  SAT. 
Respondents  pointed  out  that  44%  found  it  necessary  to  abbreviate  the  SAT  process.  In  fact, 
when  the  question  was  asked  slightly  differently,  i.e.,  is  there  a  need  to  accelerate  the 
Instructional  System  Design  process,  65.1%  found  a  need  to  accelerate  the  process.  This  large 
percentage  indicates  that  a  change  in  the  process  is  necessary. 

Despite  their  dissatisfaction,  respondents  indicated  a  loyalty  to  the  SAT  process. 
Examples  of  these  comments  include  “SAT  process  is  not  time  driven,  but  event  driven.  It  must 
be  resourced  to  work.”  “SAT  is  not  the  problem,  PPBES  is  the  problem.  Resourcing  for 
equipment,  students  and  instructors  severely  impacts  SAT!” 

That  only  57%  of  the  Civilian  Training  Leaders  who  responded  to  the  survey  believe  SAT 
is  effective  in  meeting  the  Instructional  Development  needs  of  the  U.S.  Army,  is  not  strong 
support  for  the  process.  Respondents  pointed  out  that  44%  abbreviated  the  SAT  process 
whereas  65.1%  found  a  need  to  accelerate  the  process.  These  responses  point  to  significant 
challenges  to  the  viability  of  the  SAT  process. 

Despite  their  loyalty  to  the  SAT  process,  respondents  clearly  indicated  considerable 
dissatisfaction  with  the  process.  Significantly,  this  dissatisfaction  was  spread  across  all  aspects 
of  the  process.  That  roughly  half  of  all  training  developers  found  the  process  to  be  deficient  is  a 
clear  indication  that  the  current  SAT  process  requires  considerable  revision,  or  even 
replacement. 
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ALTERNATE  PROCESSES 

A  number  of  alternate  processes  exist  to  replace  or  improve  SAT.  For  the  purpose  of 
analysis  in  this  paper,  five  designs  in  Instructional  System  Design  will  be  examined:  the  Dick 
and  Carey  Model,  the  Rapid  Prototyping  Model,  the  New  Component  Design  Theory  Tutorial 
Model,  the  New  Component  Design  Theory  Experiential/Tutorial/Advisor  Model  and  the 
Elaboration  Principle. 

THE  DICK  AND  CAREY  MODEL 

The  Dick  and  Carey  Model  is  a  linear  step-by-step  model  (see  Figure  4).  The  model 
assesses  needs,  identifies  goals,  and  then  conducts  an  instructional  analysis  while  analyzing 
learners  and  the  contextual  nature  of  the  material.  The  next  step  is  to  write  the  performance 
objectives,  followed  by  developing  assessment  instruments,  instructional  strategy,  and 
instructional  material.  There  is  then  a  formative  evaluation,  which  will  result  in  revising  the 
instruction.  The  last  step  is  to  conduct  a  summative  evaluation.19  The  biggest  drawback  to  this 
model  is  its  linear,  step-by-step  process.  Dr.  Gus  Prestera,  an  instructional  design  consultant 
and  former  e-Learning  leader  with  Allen  Communications,  says  of  the  Dick  and  Carey  Model;  "In 
my  specific  context,  the  Dick  and  Carey  model  represents  a  fairy-tale,  describing  how  instruction 
could  be  developed  if  we  had  unlimited  time,  knowledge,  and  resources." 20  Additionally,  expert 
writers  on  ISD,  Wilson,  Jonassen  and  Cole  offer  criticism  of  the  Dick  and  Carey  Model  in 
Cognitive  Approaches  to  Instructional  Design:  "From  its  inception,  ID  practice  has  fallen  short  of 
its  ideal  prescription.  Based  on  cybernetic  principles  of  general  systems  theory,  the  ideal  design 
process  relies  on  constant  systemic  feedback." 21  Due  to  the  step-by-step  linear  process,  the 
Dick  and  Carey  Model  is  not  recommended  to  replace  the  SAT  process. 


FIGURE  4:  DICK  AND  CAREY  INSTRUCTIONAL  SYSTEM  DESIGN  MODEL  22 
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THE  RAPID  PROTOTYPING  MODEL 

Rapid  Prototyping  Model,  (see  Figure  5)  was  originally  a  concept  in  the  software 
development  field  but  is  now  regarded  as  a  systems  and  product  development  model.  The 
concept  of  rapid  prototyping  initiates  a  teaming  approach,  which  uses  concepts  of  spiral 
development  to  create  training.  In  assessing  the  Rapid  Prototyping  Model  Gus  Prestera 
asserts, 

Generally,  rapid  prototyping  models  involve  learners  and/or  subject  matter 
experts  (SMEs)  interacting  with  prototypes  and  instructional  designers  in  a 
continuous  review/revision  cycle.  Developing  a  prototype  is  practically  the  first 
step,  while  front-end  analysis  is  generally  reduced  or  converted  into  an  on-going, 
interactive  process  between  subject-matter,  objectives  and  materials. 23 

Wilson,  Jonassen  and  Cole  describe  the  Rapid  Prototyping  Model  as  "In  a  design 
process,  early  development  of  a  small-scale  prototype  used  to  test  out  certain  key  features  of 
the  design.  Most  useful  for  large-scale  or  projects." 24  ISD  writers,  Edmonds,  Branch  and 
Mukherjee,  however  warn  that  the  Rapid  Prototyping  Model  should  be  used  only  by  experienced 
Instructional  Designers.  They  also  criticize  the  model  as  a  capitulation  to  time  demands. 25 

The  major  weakness  of  the  Rapid  Prototyping  Model  is  that  it  does  not  have  a  formative  or 
summative  evaluation.  This  weakness  in  the  model  could  be  overcome  by  the  addition  of 
constant  evaluation  and  feedback  loops.  With  the  addition  of  evaluation  in  this  model  and  an 
effective  training  program  the  Rapid  Prototyping  Model  is  recommended  as  a  possible 
replacement  of  the  SAT  process.  This  model  could  replace  the  SAT  process. 


Assess  Needs 
&  Analyze  Content 

Set  Objectives 

Construct  Prototype  (Design) 

Utilize  Prototype  (Research) 

] 

Install  &  Maintain  System 

Time 


FIGURE  5:  RAPID  PROTOTYPING  MODEL26 
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THE  NEW  COMPONENT  DESIGN  THEORY  TUTORIAL  MODEL 

The  New  Component  Design  Theory  Tutorial  Model  (see  Figure  6)  is  a  Computer  Assisted 
Instruction  Model  based  on  the  Branching  Programmed  Instruction  Model.  The  Tutorial  Model 
is  both  an  instruction  and  instructional  design  model.  The  model  presents  a  page  of  text  to  a 
learner  to  study.  After  study,  the  model  asks  a  question  of  the  learner.  If  the  learner’s  response 
is  correct,  the  model  provides  positive  feedback.  If  the  learner’s  response  is  incorrect,  the 
model  provides  feedback  and  remedial  material,  and  the  cycle  is  repeated.  This  questioning 
technique  is  known  as  the  Socratic  Method  and  is  a  basis  of  teaching  and  learning.27  The 
advantage  of  this  model  is  its  simplicity  and  its  ability  to  teach.  One  major  disadvantage  is  that 
many  things  do  not  lend  themselves  to  this  type  of  teaching,  for  example  teaching  how  to 
operate  machinery,  trouble  shooting  electrical  circuits,  and  designing  a  piece  of  equipment.  The 
Tutorial  Model  is  a  great  instructional  tool,  however  due  to  its  severe  limitations  as  indicated 
above  the  model  is  not  recommended  to  replace  the  SAT  process. 
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FIGURE  6:  TUTORIAL  MODEL28 

EXPERIENTIAL/TUTORIAL/ADVISOR  MODEL 

The  Experiential/Tutorial/Advisor  Model  (see  Figure  7)  uses  computer  technology  as  a 
method  in  the  design  of  instruction.  As  with  the  Tutorial  Model,  the  Experiential/Tutorial/Advisor 
Model  is  both  an  instructional  model  and  instructional  design  model.  The  technology  can  be 
used  as  a  tutor  and  as  a  simulation.  The  simulation  is  a  micro-world  that  the  learner  can 
influence  and  scrutinize.  Almost  all  instruction  can  be  simulated  and  the  learner  can  be  given 
control  over  the  simulation.  The  computer  can  be  a  mentor  that  monitors  learner  performance 
providing  help  and  assistance  when  needed.  The  computer  can  perform  many  things 
simultaneously  with  the  most  effective  learning  coming  from  the  interaction  of  the  learner  with 
the  subject  matter  in  multiple  learning  modalities,  such  as  observation,  active  experimentation, 
Socratic  Dialogue,  and  simulation. 

This  model  is  a  basis  of  what  is  called  discovery  learning.  Discovery  learning  is 
constructionist  method  of  instruction,  which  is  a  concept  in  designing  education  for  adults.  This 
model  reinforces  adult  learning  principles  articulated  by  Malcom  Knowles.29 

The  model  is  best  used  to  teach  processes,  but  it  can  be  applied  to  almost  all  material. 
This  is  a  wonderful  instructional  design  model  and  is  used  extensively  in  interactive  multi-media 
instruction  and  computer  assisted  simulation  training.  The  major  shortfall  of  this  model  is  the 
time  needed  to  develop  this  micro-world  with  all  the  branches  and  sequels  necessary  for  a 
vibrant  learning  experience.  This  model  can  take  appreciably  more  time  to  develop  instruction 
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than  the  SAT  process.  Therefore,  this  model  cannot  be  recommended  to  replace  the  SAT 
process. 


Experiential/T utorial/Advisor  Model 


FIGURE  7:  EXPERIENTIAL/TUTORIAL/ADVISOR  MODEL30 
THE  ELABORATION  PRINCIPLE 

The  Elaboration  Principle  (Figure  8)  is  a  model  and  a  schema  for  the  presentation  and 
design  of  instruction.  The  Elaboration  Principle  “promote[s]  incremental  elaboration  of  the  most 
appropriate  cognitive  structures  to  enable  the  student  to  achieve  increased  generality  and 
complexity  in  the  desired  learned  performance”.31  In  Elaboration,  the  subject  matter  must  be 
sequenced  and  organized  in  a  way  that  promotes  learning  in  a  gradual  way  that  increases  in 
complexity.  The  course  organization,  which  looks  like  a  graphic  organizer,  is  the  basis  on  which 
the  instruction  is  built.  Instruction  usually  consists  of  modules,  each  module  is  composed  of 
lessons,  and  lessons  are  composed  of  learning  activities.  The  path  through  the  course 
organization  can  be  left  to  the  learner  or  can  be  determined  by  the  instructional  designer. 
Tracking  mechanisms  are  used  to  follow  the  learner  through  the  instruction. 


17 


Elaboration  is  an  effective  design  method.  The  material  can  be  structured  in  sequences 
that  assist  the  student  to  learn  prerequisite  material  first.  The  method  then  would  allow  the 
student  to  go  into  more  depth  in  a  particular  area.  This  ability  to  drill  down  to  the  details  is  a 
popular  method  in  web-site  design.  This  method  is  quite  unstructured  and  depends  heavily  on 
the  subject  matter  expertise  of  the  developer.  However,  such  subject  matter  expertise  normally 
is  nonexistent  with  new  weapons  and  equipment.  This  method,  therefore,  is  not  recommended 
to  replace  SAT  process  at  this  time. 


FIGURE  8:  ELABORATION  PRINCIPLE32 

COURSES  OF  ACTION  AND  ANALYSIS 

Of  the  several  Instructional  System  Design  processes  just  discussed,  only  the  Rapid 
Prototyping  Model  (with  a  formative  and  summative  evaluation  added  to  the  model)  appears 
feasible.  Another  alternative  is  to  continue  to  use  the  existing  SAT,  but  to  adapt  it,  perhaps 
considerably.  The  discussion  that  follows  will  examine  eight  possible  options  for  adapting  the 
current  SAT  process  that  were  addressed  to  Civilian  Training  Leaders  that  focused  on 
increasing  the  speed  of  delivery  of  training  products  to  soldiers. 
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Courses  of  Action 

Percent  of  Civilian  Training  Leaders 
Recommending 

1 .  Increase  the  Education  and  Training 
of  the  T raining  Developers. 

20% 

2.  Increase  the  funding  to  develop 
training  products. 

22% 

3.  Abbreviate  or  change  the  SAT 
process. 

6% 

4.  Contract  out  parts  of  the  Instructional 
System  Design  Process 

7% 

5.  Hire  more  Training  Developers 

21% 

6.  Create  multi-disciplinary  teams  to 
implement  the  SAT  Process 

15% 

7.  Use  a  different  Instructional  System 
design  process 

3% 

8.  Teach  the  Military  Training  Leaders 
to  effectivelv  imnlement  the  SAT  nrncess. 

36% 

(Note:  Responses  will  not  add  up  to  1 00%  because  respondents  could  choose  multiple 
answers.) 


TABLE  5:  CIVILIAN  LEADERS  OPINIONS 

INCREASE  THE  EDUCATION  AND  TRAINING  OF  THE  TRAINING  DEVELOPERS 

As  indicated  in  Table  5,  above,  20%  of  respondents  to  the  SAT  Questionnaire  pointed  out 
that  increasing  the  education  and  training  of  the  training  developers  would  speed  the  production 
of  training  to  soldiers.  Increasing  the  knowledge  and  training  that  the  developer  receives  will 
enhance  the  output  and  productivity  to  the  soldier.  This  is  a  good  option  that  over  time  will 
enhance  productivity. 

A  key  question,  however,  remains:  how  to  ensure  that  in  a  period  of  constrained  time, 
dollars  and  personnel,  the  appropriate  personnel  can  actually  receive  such  training?  Reporting 
on  the  training  qualifications  of  the  civilian  training  developers  would  be  an  initial  step  in 
understanding  the  scope  of  the  training  requirement.  Individual  T raining  Plans,  which  are 
submitted  to  career  program  managers  at  each  school  and  consolidated  at  TRADOC 
DCSOPS&T,  would  indicate  the  level  of  training  developers’  competency  and  proficiency. 
Evaluation  of  the  education  and  training  of  training  developers  could  be  accomplished  by  the 
Quality  Assurance  Office  at  each  installation  or  proponent  school. 

INCREASE  THE  FUNDING  TO  DEVELOP  TRAINING  PRODUCTS 

Twenty  two  percent  of  the  Civilian  T raining  Leaders  who  responded  to  the  questionnaire 
indicated  that  increasing  funding  to  develop  training  products  would  speed  up  the  process.  The 
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increase  in  funding  to  develop  training  products  directly  relates  to  personnel  authorizations. 
Anecdotally,  increases  the  in  TRADOC  budget  in  the  past  have  not  positively  affected  training 
development.  Since  manpower  is  the  major  cost  in  training  development,  a  reduction  in  civilian 
personnel  budget  has  a  dramatic  impact  on  training  development.  The  GAO  Report,  Army 
Needs  to  Address  Resource  and  Mission  Requirements  Affecting  Its  Training  and  Doctrine 
Command  Feb  2003stated,  “Likewise,  the  increase  in  funding  was  not  uniform  across  TRADOC 
budget  categories.  For  example,  between  fiscal  years  1 995  and  2001 ,  funding  in  the  civilian 
personnel  budget  category  decreased  by  $60  million,  while  contract  funding  increased  by  $31 7 
million.  The  decrease  in  civilian  personnel  funding  resulted  in  the  loss  of  personnel 
authorizations.’®3  The  authorization  problem  is  in  the  process  of  being  corrected. 

Current  TRADOC  manpower  authorizations  show  increases  in  personnel  assigned  to 
training  development  in  FY  04  with  931 ,  FY  05  with  1 080,  and  FY  06  with  1 075.  The  increase  in 
authorizations,  if  they  result  in  actual  assignments,  will  certainly  help  address  the  problem.  This 
problem  is  illuminated  in  the  GAO  Report  Army  Needs  to  Address  Resource  and  Mission 
Requirements  Affecting  Its  Training  and  Doctrine  Command  Feb  2003 stated; 

They  noted  that  in  several  instances  the  Army  provided  funding  for 
transformation  related  activities  but  did  not  provide  the  associated  personnel 
authorizations.34 

ABBREVIATE  OR  CHANGE  THE  SAT  PROCESS 

Six  percent  of  the  Civilian  Training  Leaders  indicated  they  believe  abbreviating  or 
changing  the  SAT  process  would  speed  up  training  development.  Interestingly,  however,  44.2% 
questionnaire  respondents  indicated  that  they  had  abbreviated  the  SAT  process  in  the  past  6 
months.  The  inconsistency  can  be  explained  in  two  ways.  First,  the  pressure  to  deliver  training 
products  due  to  mission  requirements  is  causing  Civilian  Training  Leaders  to  abbreviate  the 
process.  Second,  the  resistance  to  abbreviation  is  due  partly  to  normal  human  resistance  to 
change  and  the  loyalty  to  the  organization  to  produce  the  best  training  product  for  the  soldier 
and  the  Army. 

There  are  drawbacks  to  this  option.  For  example,  abbreviation  could  result  in  an 
incomplete  task  analysis,  which  might  omit  a  step  in  a  procedure  thereby  leading  to  damaging 
equipment  or  placing  soldiers  at  risk. 

To  mitigate  these  risks,  TRADOC  could  adopt  a  Rapid  Prototyping-type  development 
model.  This  could  shorten  the  time  for  fielding  training  products.  This  model  could  be  tested  at 
proponent  schools  by  a  formal  experiment,  which  would  compare  the  development  of  a  training 
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product  using  both  processes,  (SAT  and  Rapid  Prototyping)  and  determine  which  process 
delivers  a  better  training  product  quicker. 

CONTRACT  OUT  PARTS  OF  THE  INSTRUCTIONAL  SYSTEM  DESIGN  PROCESS 

Seven  percent  of  the  Civilian  T raining  Leaders  believe  that  contracting  out  parts  of  the 
ISD  process  would  speed  the  development  of  training  products.  One  problem  with  contracting 
out  portions  of  the  process  is  the  need  to  supervise  the  delivery  of  the  products.  The 
Contracting  Officer  Representative  who  approves  the  delivery  of  the  products  normally  does  not 
have  the  expertise  to  know  what  a  good  training  product  is.35  The  delivery  schedule  and  review 
process  is  normally  so  constrained  that  the  reviews  are  usually  not  thoroughly  completed. 
Personal  experience  with  the  process  has  reinforced  this  point,  as  a  representative  comment 
from  one  Civilian  Training  Leaders  indicates: 

Currently,  I  am  reviewing _ products  for _ delivered  by  a  contractor. 

The  contract  had  been  let  and  work  had  started  by  the  time  I  reported  for  work.  I 

know  that  they  had  to  take  some  short  cuts  because  they  wanted _ tasks 

converted  to _ in  six  months,  i  don’t  know  what  your  experience  has  been 

with  contractors,  but  I  have  not  seen  many  products  that  are  worth  the  money  the 
government  is  paying.  It  is  a  multifaceted  problem.  The  product  will  only  be  as 
good  as  the  Government  Furnished  Information  (GFI).  The  GFI  is  often 
developed  without  the  proper  analysis;  frequently  because  the  training  developer 
does  not  know  how  to  do  it,  there  is  no  emphasis  on  it,  or  because  there  is  not 
sufficient  time.  Time  of  course  is  another  contributor.  Often  it  is  a  matter  of 
getting  lessons  learned  out  ASAP  to  those  soldiers  being  deployed.36 

In  2001 ,  TRADOC  had  1 609  contracted  training  developers  compared  to  886  Department 
of  the  Army  civilian  training  developers.  The  TRADOC  cost  differential  for  contracted  training 
developer  is  $60.1 0  an  hour  to  $39.00  an  hour  for  Department  of  the  Army  civilian  training 
developer.37  Despite  the  drawbacks,  the  use  of  contracting  can  add  to  Training  Development 
capacity  in  the  short  term.  The  need  for  close  supervision  and  adequate  quality  control  of 
contracted  products  is  essential  to  make  contracting  worthwhile.  One  also  must  recognize  that 
such  a  level  of  oversight  will  take  training  developers  away  from  their  primary  duties,  potentially 
exacerbating  problems  the  contracting  out  was  supposed  to  solve. 

HIRE  MORE  TRAINING  DEVELOPERS 

Twenty  one  percent  of  the  Civilian  Training  Leaders  indicated  the  solution  to  the  problem 
is  to  hire  more  Training  Developers.  Reductions  of  Training  Development  personnel  over  the 
last  decade  are  a  major  reason  for  the  shortfall  in  the  quality  and  quantity  of  Training  Products 
produced  at  TRADOC  Schools.  The  GAO  Report38  confirms  this  problem: 
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Moreover,  the  Command  [TRADOC]  has  workload  backlogs  in  other  mission 
areas,  such  as  developing  training  materials  and  Army  doctrine. ..Furthermore, 
two  Army  leadership  panels  concluded  that  TRADOC’s  training  and  development 
standards  had  deteriorated,  and  mechanisms  for  evaluating  training  and  leader 
development  programs  were  lacking.39 

Any  solution  to  the  problem  should  include  hiring  full-time  Army  Civilian  Employees  who  can 
perform  the  Training  Development  task. 

CREATE  MULTI-DISCIPLINARY  TEAMS  TO  IMPLEMENT  THE  SAT  PROCESS 

The  creation  of  multi-disciplinary  teams  to  implement  the  SAT  process  garnered  support 
from  1 5%  of  respondents.  The  use  of  multi-disciplinary  teams  in  the  development  of  Distance 
Learning  Training  products  has  been  very  effective  in  a  number  of  TRADOC  Schools.40  The 
successes  were  reinforced  by  TRADOC  DCSOPS&T  giving  the  U.S.  Army  Engineer  School 
special  permission  to  establish  a  multi-disciplinary  team  to  develop  training  products.  This 
option  can  be  highly  successful  with  appropriate  teaming  and  leadership.  DCSOPS&T  has 
been  willing  to  grant  this  permission  upon  request  and  recommended  it  in  TRADOC  Regulation 
350-70.  This  represents  an  avenue  that  should  be  pursued. 

USE  A  DIFFERENT  INSTRUCTIONAL  SYSTEM  DESIGN  PROCESS 

Only  three  percent  of  the  Civilian  Training  Leaders  indicated  that  using  a  different 
Instructional  System  Design  would  speed  up  the  development  of  training  products.  However, 
their  satisfaction  may  be  due  to  their  comfort  with  the  status  quo,  which  may  not  be  useful  in  the 
long-term.  A  comment  from  one  of  the  Civilian  Training  Leaders  is  an  example, 

I  wholeheartedly  endorse  the  SAT  process  because  it’s  a  tried,  proven  process 
that  produces  top  quality  training  products  and  programs.  However,  continued 
reliance  on  Army/TRADOC  Schools,  and  their  Training  Developers,  will  spell 
disaster  for  future  Army  Training.  Current,  rapid  technology  and  spiral 
development  is  eating  our  lunch.  We  must  ensure  systems  procurement 
contracts  include  appropriate,  simultaneous  training  development  IAW  the  SAT 
process  if  we’re  to  keep  up  and  maintain  currency  and  relevancy.  Meanwhile, 
we’re  falling  more  behind  each  passing  day. 

As  indicated  earlier,  the  Army  should  support  experiments  at  a  number  of  schools 
comparing  the  Rapid  Prototyping  Model  or  other  models  to  the  SAT  Model  in  the  development 
of  training  products.  Because  most  Civilian  T raining  Leaders  in  the  Army  have  little  to  no 
experience  with  other  Instructional  System  Design  processes,  this  experimentation  should  be 
overseen  by  the  Army  Research  Institute.  This  analysis  should  be  completed  as  soon  as 
possible. 
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TEACH  MILITARY  TRAINING  LEADERS  TO  EFFECTIVELY  IMPLEMENT  THE  SAT 
PROCESS 

Thirty  six  percent  of  the  Civilian  Training  Leaders  indicated  that  teaching  Military  Training 
Leaders  the  SAT  process  would  significantly  improve  implementation  of  the  process.  This 
observation  identifies  the  problem  of  the  revolving  door  that  many  Military  Training  Leaders 
have  been  experiencing.  Specifically,  the  rotation  of  Directors  of  Training  at  many  TRADOC 
Schools  needs  attention.  During  my  ten  years  of  experience  in  TRADOC  Schools,  I  have  not 
observed  a  Director  of  Training  spending  more  than  eighteen  months  in  the  position.  Moreover, 
Military  Training  Leaders  normally  have  little  formal  education  in  the  process  they  are  leading. 
Despite  the  fact  that  courses  are  available  to  teach  Senior  Leaders  about  the  process,  few  have 
the  time  or  inclination  to  take  the  courses.  TRADOC  should  require  School  Commandants  to 
ensure  Military  Training  Leaders  undergo  this  training. 

CONCLUSION 

The  realities  of  the  Global  War  on  Terrorism  and  the  responses  of  Civilian  Training 
Leaders  clearly  indicate  that  the  SAT  process  needs  improvement.  Although  SAT  has  produced 
great  training  products  for  over  30  years,  it  is  an  industrial  age  model.  Unfortunately,  the  time  is 
past  for  the  lock  step  methods  that  produced  highly  competent,  behaviorally  trained  soldiers. 

The  lack  of  funding  and  personnel  to  perform  the  function  only  exacerbated  the  problem.  A 
new,  information  age  model  is  needed.  As  stated  earlier.  “Current,  rapid  technology  and  spiral 
development  is  eating  our  lunch.” 

RECOMMENDATION 

Stated  simply  and  directly,  the  following  actions  will  increase  the  speed  the  development 
of  training  products  in  the  U.S.  Army.  In  the  short-term:  hire  and  train  more  training  developers 
immediately.  Direct  TRADOC  Schools  to  form  the  T raining  Developers  into  multi-disciplinary 
teams  and  use  automated  tools  that  will  speed  the  SAT  process  (e.g.  Designers  Edge). 
Additionally,  make  evaluation  of  the  SAT  process  part  of  the  Quality  Assurance  Office  mission. 
To  identify  a  long-term  solution,  immediately  begin  experimenting  with  Rapid  Prototyping  in  a 
head-to-head  contest  with  the  SAT. 
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APPENDIX  A  -  SYSTEMS  APPROACH  TO  TRAINING  QUESTIONNAIRE 
ARMY  RESEARCH  INSTITUTE  CONTROL  NUMBER  DAPE-ARI-AO-04-25,  RCS:  MILPC-3 


Directions 

Please  complete  all  applicable  items  in  this  questionnaire  to  the  best  of  your  abilities.  Prior  to  responding  to  each  item,  please  read 
it  carefully  while  considering  the  training  products  you  or  your  team  developed  during  the  last  6  months.  This  2-page  questionnaire 
should  take  about  1 5  minutes  to  complete. 

Select  appropriate  responses  with  checkmarks;  these  can  be  made  with  either  pen  or  pencil.  When  finished,  please  forward  the 
paper  version  of  your  questionnaire  to:  Richard  W.  Swain,  1400  Waggoners  Gap  Road,  Carlisle,  PA  17013.  If  you  have  any 
questions  or  concerns  about  this  questionnaire  please  contact  Richard  Swain  at  1  -71 7-243-8262  or  1  -915-525-9372.  Your 
individual  answers  will  be  kept  strictly  confidential.  Your  answers  will  be  combined  with  other  training  developers  to  describe  the 
current  status  of  Instructional  System  Design  in  the  United  States  Army.  If  you  would  like  a  copy  of  the  results  of  this  research 
please  contact  me  at  RickWSwain(a>aol.com  or  Richard. SwainfS) Carlisle. army. mil 

Part  One  Methods  Used  In  Development  Of  Training  Products 

Q1 .  In  the  past  6  months,  did  your  team  find  it  appropriate  to  abbreviate  or  eliminate  steps  in  the  instructional  systems  design 
process  for  most  of  the  products  developed? 

Yes 

No 

Q2.  Did  your  team  use  Designer’s  Edge,  a  software  tool  provided  by  the  US  Army  for  the  specification  of  steps  in  the  design 
process,  in  the  development  of  these  products  that  you  produced  in  the  past  6  months? 

Yes 

No 

Q3.  For  most  of  the  training  products  developed  over  the  last  6  months,  did  your  team  find  it  necessary  to  accelerate  the 
instructional  systems  design  process? 

Yes 

No 

Part  Two  :  Measuring  Effectiveness  of  Specific  Instructional  System  Design  (ISD)  Components  (Effective  meaning;  producing  or 
capable  of  producing  an  intended  result  or  having  a  striking  effect) 


Very  Effective 
Somewhat  Effective  I 
Neither  Effective  or  Ineffective  I 
Somewhat  Ineffective  I 
Very  Ineffective  I 


Q4.  How  effective  was  the  Instructional  System  Design  training 
that  you  received  in  the  Army? 

Q5.  How  effective  was  your  Analysis  portion  of  the  SAT  Process  in  your  last  development  of  a  Training  Product? 

Q6.  How  effective  was  your  Design  portion  of  the  SAT  Process  in  your  last  development  of  a  Training  Product? 

Q7.  How  effective  was  your  Development  portion  of  the  SAT  Process  in  your  last  development  of  a  T raining  Product? 
Q8.  How  effective  was  your  Implementation  portion  of  the  SAT  Process  in  your  last  development  of  a  Training  Product? 
Q9.  How  effective  was  your  Evaluation  portion  of  the  SAT  Process  in  your  last  development  of  a  T raining  Product? 


ooooo 

ooooo 

ooooo 

ooooo 

ooooo 

ooooo 


Part  Three ;  Results  from  Possible  Manipulations  of  Specific  ISD  Components 

Q1 0.  What  steps  in  the  Instructional  System  Design  Process  did  you  abbreviate  or  eliminate  during  the  last  six  months?  (check  all 
that  apply) 

?  Analysis 
?  Design 
?  Development 
?  Implementation 
?  Evaluation 
?  None  (If  none  skip  to  Q1 6) 

If  any  of  the  above  components  were  modified,  what  is  your  estimate  of  the  time  savings  gained  by  abbreviating  or 
eliminating... 
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80  +  work  hours 
40  to  80  work  hours  I 


21  to  40  work  hours  I 
11  to  20  work  hours  I 
1  to  10  work  hours  I 


Q11.  Analysis 

Mill 

ooooo 

Q12.  Design 

ooooo 

Q13.  Development 

ooooo 

Q14.  Implementation 

ooooo 

Q15.  Evaluation 

ooooo 

Part  Four:  What  is  your  opinion  on  Challenges  and  Suggestions 

Q16.  In  your  opinion  what  are  the  top  three  challenges  facing  the  SAT  (Systems  Approach  to  Training)  process  in  the  United  States 
Army?  (Choose  up  to  3) 

?  Any  problems  with  the  SAT  process  in  the  development  of  training  products  are  tiny  and  can  easily  be  overcome. 

?  Current  SAT  process  works  well,  and  the  chief  difficulty  is  lack  of  utilization  by  Training  Developers. 

?  Current  SAT  process  is  fine,  but  military  leaders  do  not  know  how  to  manage  the  process. 

?  Current  SAT  process  is  fine,  but  strict  adherence  to  regulations  causes  difficulty 

?  In  general,  SAT  is  a  military  process  more  about  rules  and  guidelines  then  product 
?  In  general,  SAT  process  is  not  adequate  for  today's  training  needs:  the  speed  of  change  in  weapons  systems 
and  training  outpace  the  SAT 

?  In  general,  SAT  process  is  designed  for  behavioral,  not  educational,  tasks 

?  There  isn’t  enough  education  and  training  on  SAT 
?  There  aren’t  enough  Training  Leaders  who  understand  the  SAT  Process 

?  There  aren’t  enough  Training  Developers  to  implement  the  process 

?  Funding  for  Training  Development  over  the  last  decade  is  inadequate 


Other  (specify) _ 

Q17.  Is  the  number  of  personnel  assigned  to  Training  Development  sufficient  to  meet  the  requirements  of  the  SAT  Process? 
?  Very  Sufficient 

?  Somewhat  Sufficient 

?  Neither  Sufficient  or  Insufficient 

?  Somewhat  Insufficient 

?  Very  Insufficient 

Q18.  How  sufficient  has  the  funding  to  meet  the  regulatory  guidance  in  TRADOC  350-70  to  produce  training  products? 

?  Very  Sufficient 

?  Somewhat  Sufficient 

?  neither  Sufficient  or  Insufficient 

?  Somewhat  Insufficient 

?  Very  Insufficient 

Q19.  What  do  you  believe  would  increase  the  speed  of  the  Instructional  System  Design  Process  (SAT  Process)  in  the  Army? 
(MARK  ALL  THAT  APPLY) 

?  Increase  Education  and  Training  of  Training  Developers 
?  Increase  Training  Development  Funding 

?  Abbreviate  the  SAT  Process 

?  Hire  more  Training  Developers 

?  Contract  out  parts  of  the  Instructional  System  Design  Process 
?  Create  multi-disciplinary  teams  to  implement  the  SAT  Process 
?  Use  a  different  Instructional  System  Design  Process 
?  Other _ 
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Demographics 


Please  provide  the  following  demographic  information.  It  will  only  be  used  to  make  statistical  comparisons  between  different  groups 
of  respondents;  it  will  not  be  used  to  profile  individual  respondents. 

Q20.  What  is  your  gender?  ?  Male  Q21.  What  is  your  age?  ?  20-29 

?  Female  ?  30-39 

?  40-49 
?  50-59 
?  59+ 


Q22.  What  is  the  highest  level  of  education  you  have  completed? 
?  Some  high  school  or  less 
?  High  school  graduate 
?  Attended  some  college 
?  Associate  degree 
?  Bachelor’s  degree 
?  Post-college  graduate 


Q23.  What  is  your  Federal  Job  Series? 

?  Instructional  System  Specialist  (1750) 

?  Training  Instruction  (1712) 

?  Instructor  (1712) 

?  Doctrine  Developer  (0301 ) 

?  General  Education  and  Training  (1701) 

?  Combat  Developer  (0301) 

?  Education  and  Training  Support  (1702) 

?  Other 
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crawl  up  to  the  observation  point  and  issue  the  CALL  FOR  FIRE.  Bracketing  was  the  only 
choice  before  the  call  Fire  for  effect.  Back  to  the  CALL  FOR  FIRE.  There  is  a  really  good 
video/film  from  the  Field  Artillery  school  that  shows  vivid  examples  of  why  bracketing  was  then 
critical.  One  line  said:  You  can  do  it  the  right  way  and  hit  the  target,  or  you  can  do  it  the  John 
Wayne  way,  and  take  incoming  mortar  fire.  I  like  process  that  can  be  trained  to  everyone.  How 
do  we  learn  about  that  process?  Somehow,  someone  has  to  figure  out  what  the  soldiers/sailors 
need  to  know  to  survive  and  thrive.  That  knowledge  has  to  be  communicated  to  those  who 
prepare/train  new  folks.  Acquisition  of  this  knowledge  is  not  obvious  and  not  easy.  Is  there  a 
better  way  of  doing  front-end  analysis?  I'll  bet  that  there  are  much  better  ways.  What  I  don't 
see  is  a  clear  connect  between  fleet  performance  and  the  training  system.  Currently  there  is  a 
new  Navy  program  that  hopes  to  address  these  issues  by  reorganization:  The  five  Vector 
Model.”  (Branson,  personal  communication,  2002) 

The  last  question  was;  What  new  thoughts  do  you  have  on  this  after  thirty  years?  “When  we 
originally  installed  the  IPISD  into  Army  schools-and  some  Navy  schools-we  learned  something 
bureaucratically  real:  The  term  evaluation  meant  the  OER,  (sic  Officer  Evaluation  Report)  not 
the  process  of  systematically  correcting  errors.  What  I  would  like  to  see  is  a  clear  definition  of 
what  sailors/soldiers  need  to  know  on  their  first  duty  assignment-call  it  ISD,  SAT-I  don't  care. 
How  do  we  know  reliably  what  these  folks  need  to  know?  So,  if  I  were  the  CNO  (sic  Chief  of 
Naval  Operation)  or  the  CSA,  (sic  Chief  of  Staff  United  States  Army)  what  would  I  do?  I  would 
demand  that  we  have  a  process  of  finding  out  what  folks  need  to  know.  When  I  know  that,  I  can 
design  a  development  system  to  make  it  happen.  You  have  raised  a  very  big  question  and  the 
resolution  to  it  must  come  from  informed  commanders.  New  commanders  MUST  make 
something  happen  differently.  The  Army  had  that  problem  as  did  the  Navy.”  (Branson, 
personal  communication,  2002) 

16  Branson,  "ISD:  Is  the  System  out  of  Control? ",  p.  20. 

16  The  U.S.  Army  War  College  and  the  Army  Research  Institute  authorized  the  questioning 
of  the  Civilian  Training  Leaders  in  the  U.S.  Army.  The  survey  instrument  was  sent  to 
participants  in  October  thru  December  2004.  A  total  of  281  survey  instruments  were  sent  out  to 
the  senior  Civilian  Training  Leaders  in  the  U.S.  Army.  The  return  rate  for  the  Systems  Approach 
to  Training  Questionnaire  was  30.6%,  giving  a  level  of  confidence  of  95%. 

17  These  observations  were  corroborated  by  the  responses  to  questions  1 7  and  1 8,  which 
are  outlined  in  Table  3  and  4. 

18  Andy  Washko,  Deputy  Director  DOTD,  interview  by  author  1  May  2004,  Fort  Bliss,  TX. 

19  The  difference  between  a  formative  and  summative  evaluation  is  the  designer  conducts 
the  formative  evaluation  and  an  independent  evaluator  conducts  the  summative  evaluation. 
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20  Gus  Prestera,  “Rapid  Prototyping  Model;  Instructional  Design  Models”,  22  March  2004; 
available  from  <http://www.personal.psu.edu  /users/g/e/gepl  1 1/html/M4/L1%20- 
%20ISD/M4L1  PI  .htm#rapid_proto>. Internet;  accessed  18  May  2004,  para  4. 

21  Brent  Wilson,  David  Jonassen,  and  Peggy  Cole,  "Cognitive  Approaches  to  Instructional 
Design."  in  The  Astd  Handbook  of  Instructional  Technology,  ed.  G.M.  Piskurich,  21.1-21.22. 
(New  York:  McGraw-Hill,  1 993), pp.  21-22. 

22  Clark, “ISD  Big  Dog  Site  Donald  Clark 

23  Prestera,  “Rapid  Prototyping  Model;  Instructional  Design  Models”,  22  March  2004  ;para  1 . 

24  Ibid.,  p.18. 

25  Gerald  S.  Edmonds,  R.C.  Branch,  &  P.  Mukherjee,  "A  Conceptual  Framework  for 
Comparing  Instructional  Design  Models."  Educational  Technology  Research  and  Development 
42,  no.  4  (1994):  55-72. 

26  Steven  D.  Tripp,  &  Barbara  Bichelmeyer,  "Rapid  Prototyping:  An  Alternative  Instructional 
Design  Strategy."  Educational  Technolgy  Research  and  Design  38,  no.  1  (1990):  31-44. 

27  M. David  Merrill,  David  I.  Twitchell,  Instructional  Design  Theory,  (Englewoods  Cliff,  N.J.: 
Educational  Technology  Publications,  1994),  p.  354. 

28  Ibid.,  p.355  Fig  17.1. 

29  Adults  are  autonomous  and  self-directed,  like  to  relate  learning  to  their  experiences,  are 
goal  oriented,  are  relevancy  driven  in  regards  to  learning,  and  are  practical  learners.  This 
model  utilizes  these  principles. 

30  Ibid.,  p.  356  Fig  17.2. 

31  Ibid.,  p.  358. 

32  Ibid.,  p.  362  Fig  17.4. 

33  GAO,  Defense  Management,  p.  1 6. 

34  Ibid.,  p.  17. 

36  Andy  Washko,  Deputy  Director  DOTD,  interview  by  author  1  May  2004,  Fort  Bliss,  TX. 

36  Anonymous,  Personal  Communication  Subject  Survey,  email,  December  22,  2004 

37  Andy  Washko  Deputy  Director  DOTD  interview  by  author  1  May  2004,  Fort  Bliss,  TX. 

38  GAO,  Defense  Management,  p.  1 6. 

39  Ibid.,  p.  16. 
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40  As  the  Engineer  and  Armor  School  reported  this  during  the  March  2003  Distributed 
Learning  Conference 
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